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DETAILED ACTION 



Election/Restrictions 
1 . Applicant's election without traverse of Invention I including claims 1-94 in the 
reply filed on June 13 th , 2007 is acknowledged. 



2. Claims 95-1 04 are withdrawn from further consideration pursuant to 37 CFR 
1.142(b) as being drawn to a nonelected Invention II there being no allowable generic or 
linking claim. Election was made without traverse in the reply filed on June 13 th , 2007. 
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Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

4. Claims 1-3, 5-21, 24-32, 34-48, 49-51, 53, 55, 58-70, 73-80, 84-85 and 87-92 are 
rejected under 35 U.S.C. 102(b) as being anticipated by Video-based Multi-agent traffic 
surveillance system (hereinafter "VM"). 

As to claim 1 , VM teaches a method for detecting one or more objects belonging 
to the same object class (Abstract) comprising the steps of: a), receiving a video 
sequence comprised of a plurality of image frames (page 457, right-hand column lines 
16-23); b). applying one or more classifiers to detect components of objects in an 
image frame in the video sequence (page 459, left-hand column, lines 8-37); c). 
computing a confidence score based in part on the response from the one or more 
component detectors (page 459, left-hand column, lines 8-16); d). repeating steps b). 
and c). to detect components of objects belonging to the same object class in additional 
images frames in the video sequence (page 459, left-hand column, lines 8-28); and e). 
accumulating confidence scores from the component detectors to determine if an object 
is detected (page 459, left-hand column, lines 8-37). 
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As to claim 49, it is the system claim of claim 1 . Please see above for detail 
analysis. 

As to claim 2, VM teaches wherein if accumulated confidence scores indicate 
high confidence of a presence of an object, the method further comprising the step of: 
identifying the detected components to be an object of a particular type (page 459, left- 
hand column lines 22^25). 

As to claim 3, VM teaches wherein the object class is a vehicle (page 459, left- 
hand column lines 12-14). 

As to claim 4, VM teaches if an object is detected, outputting a detection signal 
and object position (page 459, left-hand column lines 43-58). 

As to claim 5, VM teaches testing geometry constraints on a spatial arrangement 
of detected components in an image; and applying whole-appearance classifiers an 
image patch that contains the detected components and which is aligned according to 
the position of the detected components (page 459, left-hand column lines 26-29). 

As to claim 6, VM teaches wherein the geometry constraints are derived from 
camera parameters (page 459, left-hand column lines 20-21). 
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As to claim 7, VM teaches wherein the geometry constraints are derived from 
object size (page 459, left-hand column line 27). 

As to claim 8, VM teaches wherein the geometry constraints are derived from a 
location of an object appearance in the image (page 459, left-hand column line 46). 

As to claim 9, VM teaches wherein the whole appearance classifiers detect entire 
or partial object appearance, the entire or partial object appearance being aligned 
according to positioning of at least two components (page 459, left-hand column lines 
26-29). 

As to claim 10, VM teaches wherein the component classifiers include classifiers 
for detecting components at multiple scales (page 459, left-hand column lines 30-36). 

As to claim 1 1 , VM teaches wherein component classifiers are defined by 
. discriminant features and decision rules which are learned through boosted training 
(page 459, left-hand column lines 37-38). 

As to claim 12, VM teaches wherein the discriminant features include corners 
(page 461, fig. 4 shows an object that's been identified and the system forming a box 
around the object, the box has corners). 
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As to claim 13, VM teaches wherein the discriminant features include horizontal 
edges (page 461, fig. 4 shows an object that's been identified and the system forming a 
box around the object, the box has horizontal edges). 

As to claim 14, VM teaches wherein the discriminant features include vertical 
edges (page 461 , fig. 4 shows an object that's been identified and the system forming a 
box around the object, the box has vertical edges). 

As to claim 15, VM teaches wherein the discriminant features include horizontal 
stripes (page 461 , fig. 4 shows an object that's been identified and the system forming a 
box around the object, the box may contain horizontal stripes). 

As to claim 16, VM teaches wherein the discriminant features include vertical 
stripes (page 461 , fig. 4 shows an object that's been identified and the system forming a 
box around the object, the box may contain vertical stripes). 

As to claim 17, VM teaches wherein the discriminant features include diagonal 
stripes (page 461 , fig. 4 shows an object that's been identified and the system forming a 
box around the object, the box may contain diagonal stripes). 

As to claim 20, VM teaches wherein the one or more classifiers include 
overlapping component classifiers (page 461 , fig. 4 shows an object that's been 
identified and the system forming a box around the object). 
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As to claim 21, VM teaches wherein the overlapping component classifiers 
comprises four corners representing a rear profile of a vehicle (page 461 , fig. 4 shows 
an object that's been identified and the system forming a box around the object, where 
the object is a vehicle driving on a road). 

As to claim 22, VM teaches wherein the overlapping component classifiers 
comprises four corners representing a frontal profile of a vehicle (page 461 , fig. 4 shows 
an object that's been identified and the system forming a box around the object, where 
the object is a vehicle driving on a road). 

As to claim 23, VM teaches wherein the overlapping component classifiers 
comprises four corners representing a side profile of a vehicle (page 461 , fig. 4 shows 
an object that's been identified and the system forming a box around the object, where 
the object is a vehicle driving on a road). 

As to claim 24, VM teaches wherein one of the overlapping component classifiers 
detects the bottom left corner of a vehicle (page 461 , fig. 4 shows an object that's been 
identified and the system forming a box around the object, where the object is a vehicle 
driving on a road). 
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As to claim 25, VM teaches wherein one of the overlapping component classifiers 
detects the bottom right corner of a vehicle (page 461 , fig. 4 shows an object that's 
been identified and the system forming a box around the object, where the object is a 
vehicle driving on a road). 

As to claim 26, VM teaches wherein one of the overlapping component classifiers 
detects the top left corner of a vehicle (page 461 , fig. 4 shows an object that's been 
identified and the system forming a box around the object, where the object is a vehicle 
driving on a road). 

As to claim 27, VM teaches wherein one of the overlapping component classifiers 
detects the top right corner of a vehicle (page 461 , fig. 4 shows an object that's been 
identified and the system forming a box around the object, where the object is a vehicle 
driving on a road). 

As to claim 28, VM teaches wherein positioning of the four corners of the rear 
profile for a vehicle differ for different classes of vehicles (page 461 , fig. 4 shows an 
object that's been identified and the system forming a box around the object, where the 
object is a vehicle driving on a road). 

As to claim 29, VM teaches a class of vehicle includes sedans (car; page 459, 
left-hand column lines 12-14). 
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As to claim 33, VM teaches wherein a class of vehicle includes trucks (truck; 
page 459, left-hand column lines 12-14). 

As to claim 34, VM teaches wherein a distance between any two corners of the 
vehicle is constrained (page 461, fig. 4 shows an object that's been identified and the 
system forming a box around the object, where the object is a vehicle driving on a road). 

As to claim 35, VM teaches wherein the constraint between any two corners of 
the vehicle is scaled based on a distance between the vehicle and a camera capturing 
the video sequence and camera parameters (page 459, left-hand column lines 18-21). 

As to claim 37, VM teaches wherein the accumulated confidence scores are 
inferred from confidence scores across multiple frames using a recursive filter (page 
459, left-hand column lines 49-50; page 459, right-hand column lines 1-2). 

As to claim 38, VM teaches wherein when the accumulated confidence score is a 
linear combination of the confidence scores of multiple component classifiers and the 
whole-appearance classifiers (page 459, left-hand column lines 22-25). 
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As to claim 39, VM teaches wherein when the confidence score for a principal 
component classifier is sufficiently high, the confidence score of the remaining 
component classifiers and the whole-appearance classifier are computed (page 459, 
left-hand column lines 15-21). 

As to claim 40, VM teaches wherein if an object is detected the method 
comprising the step of: tracking the object over subsequent image frames (page 459, 
right-hand column lines13-20). 

As to claim 41 , VM teaches wherein the step of tracking the object further 
comprises the step of: restricting an area of search in each subsequent image frame 
based on the location of the object in a current image frame (page 459, right-hand 
column lines 24-31). 

As to claim 42, VM teaches wherein the step of tracking the object further 
comprises the step of: determining the optimal classifier scale based on a distance 
between the object and a camera detecting the object and camera parameters (page 
459, right-hand column lines 32-41 teaches the prediction process using speed, 
however, speed is the combination of time and distance, therefore there is the need to 
use the distance parameter). 
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As to claim 45, VM teaches wherein an object class includes pedestrians 
(person; page 459, left-hand column lines 12-14). 

As to claim 47, VM teaches wherein an object class includes motorcycles 
(motorbike; page 459, left-hand column lines 12-14). 

As to claims 49-51, 53, 55, 58-70, 73-80, 84-85 and 87-92, they are the system 
claims of claims 1 , 3, 45, 47, 2, 5-17, 20-21 , 24-29, 34-35 and 37-42. Please see above 
for detail analysis. 

As to claim 57, VM teaches a display for displaying the video sequences (Fig. 4 
is a sample of a display window of the system). 
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Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 18-19, 30-32, 46 and 48 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over VM. 

As to claim 18-19, VM teaches a video-based traffic surveillance system. VM 
does not expressly disclose that the system is adapted for online usage. However, 
Examiner takes Official Notice that putting a system for online usage is well known in 
the art. It would have been obvious at the time of the invention was made to one of 
ordinary skill in the art to modify the video surveillance system and use it for an online 
system since Examiner takes official notice that it would be useful to have a surveillance 
system accessible online. 

As to claims 30-32, 46 and 48, VM teaches the classification may have one of 
the following values: car, truck, motorbike, person, toolarge and mobile_object (page 
459, left-hand column lines 10-14). VM does not expressly disclose that the classifier 
also includes bicycles, sports utility vehicles, vans, tractor-trailers and traffic signs. 
However, Examiner takes Official Notice that the non-disclosed classifiers are well 
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known in the art. It would have been obvious at the time of the invention was made to 
one of ordinary skill in the art to add the additional classifiers since Examiner takes 
official notice that the non-disclosed classifiers are common on a normal road. 

As to claims 52, 54, 71, 72 and 81-83, they are the system claims of claims 46, 
48, 18-19, 30-32. Please see above for detail analysis. 

7. Claims 36, 43 and 44, 56-57, 86 and 93-94 are rejected under 35 U.S.C. 1 03(a) 
as being unpatentable over VM in view of Brady et al. (5,761,326 hereinafter "Brady"). 

As to claim 36, VM does not teach wherein an image pyramid of multiple 
resolutions is used to detect objects of size 2*x, 4*x and so on with the classifier for the 
size x. Brady teaches a method for machine vision classification and tracking using 
multiple resolution of a region of interest that has a sample kernel of 2x2, even though 
this method only give one sample it is not limited to kernel size 2x2 (Col. 12, lines 34- 
45). Thus, Brady's machine vision method reads on the claimed pyramid multiple 
resolution method to detect objects. Therefore, it would have been obvious to one 
ordinarily skilled in the art at the time the invention was made to combine VM's 
surveillance system with Brady's multiple resolution system in order to increase the 
speed of processing the information for tracking (Brady Col. 12, lines 34-35). 
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As to claim 43, VM does not teach wherein the confidence scores of component 
classifiers are computed in a coarse to fine framework. Brady teaches a method for 
machine vision classification and tracking that searches a target region first at a lower 
resolution then at a higher resolution (Col. 12, lines 43-46). Thus, Brady's searching a 
target region from low to high resolution reads on the claimed coarse to fine framework. 
Therefore, it would have been obvious to one ordinarily skilled in the art at the time the 
invention was made to combine VM's surveillance system with Brady's multiple 
resolution system in order to increase the speed of processing the information for 
tracking (Brady Col. 12, lines 34-35). 

As to claim 44, VM does not teach wherein detection is performed on an image 
pyramid of multiple resolutions. Brady teaches a method for machine vision 
classification and tracking using multiresolution pyramid (Col. 12, lines 41-43). Thus, 
Brady's multiresolution pyramid reads on the claimed image pyramid of multiple 
resolutions. Therefore, it would have been obvious to one ordinarily skilled in the art at 
the time the invention was made to combine VM's surveillance system with Brady's 
multiple resolution system in order to increase the speed of processing the information 
for tracking (Brady Col. 12, lines 34-35). 

As to claim 56, VM does not teach wherein if an object is detected, the processor 
outputs a warning signal. Brady teaches the graphical user interface automatically 
displays alarm information when an incident has been detected (Col.4, lines 5-7). Thus, 
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Brady's alarm information reads on the claimed warning signal. Therefore, it would 
have been obvious to one ordinarily skilled in the art at the time the invention was made 
to combine VM's surveillance system with the alarm notification in order to notify the 
user. 

As to claims 86 and 93-94, they are the system claims of claims 36 and 43-44. 
Please see above for detail analysis. 
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